Skip to content

Append commit instead of individual transactions to commitlog#4404

Merged
clockwork-labs-bot merged 30 commits intomasterfrom
kim/commitlog/append-commit-reopen
Mar 3, 2026
Merged

Append commit instead of individual transactions to commitlog#4404
clockwork-labs-bot merged 30 commits intomasterfrom
kim/commitlog/append-commit-reopen

Conversation

@kim
Copy link
Copy Markdown
Contributor

@kim kim commented Feb 23, 2026

Re-open #4140 (reverted in #4292).

The original patch was merged a bit too eagerly.
It should go in after 2.0 is released with some confidence.

kim added a commit that referenced this pull request Feb 24, 2026
Cherry-picked from #4404
Better default for confirmed reads (which are the default since #4390).
kim added a commit that referenced this pull request Feb 24, 2026
github-merge-queue bot pushed a commit that referenced this pull request Feb 24, 2026
Cherry-picked from #4404
Better default for confirmed reads (which are the default since #4390).
kim added a commit that referenced this pull request Feb 26, 2026
Instead of periodic flush + sync, the simplified commitlog buffering
makes it easy to just pop a batch of transaction from the queue, commit
them, and then flush + sync after each batch.

This may lead to an overall higher number of fsyncs on the system, but
also adapts to the tx throughput of the individual database (up to the
batch size).

After taking some measurements, we may want to make the batch size
configurable at runtime.

Depends-on: #4404
@kim kim mentioned this pull request Feb 26, 2026
@Shubham8287
Copy link
Copy Markdown
Contributor

Is this same patch as reverted one or anything else?

Instead of periodic flush + sync, the simplified commitlog buffering
makes it easy to just pop a batch of transaction from the queue, commit
them, and then flush + sync after each batch.

This may lead to an overall higher number of fsyncs on the system, but
also adapts to the tx throughput of the individual database (up to the
batch size).

After taking some measurements, we may want to make the batch size
configurable at runtime.

Depends-on: #4404
github-merge-queue bot pushed a commit that referenced this pull request Feb 28, 2026
# Description of Changes

This does 3 things to the keynote-2 benchmark:
- it changes the default alpha to 1.5, which we actually tested the
other services with
- it turns on confirmed reads (not the default for < 2.0)
- it removes warmup

This was tested on
#4404 to have no impact
on the TPS throughput of spacetimedb.
**This PR shouldn't be merged before #4404 has.**

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing

This tweaks a bench test.
@kim
Copy link
Copy Markdown
Contributor Author

kim commented Feb 28, 2026

Is this same patch as reverted one or anything else?

It's the same.

Well, it's no longer the same, because #4478 got merged into this one.
I intended this one to merge, then #4478, not squash both.

@clockwork-labs-bot clockwork-labs-bot added this pull request to the merge queue Mar 3, 2026
Merged via the queue into master with commit 17cc15e Mar 3, 2026
32 of 33 checks passed
Centril added a commit that referenced this pull request Mar 3, 2026
# Description of Changes

This does 3 things to the keynote-2 benchmark:
- it changes the default alpha to 1.5, which we actually tested the
other services with
- it turns on confirmed reads (not the default for < 2.0)
- it removes warmup

This was tested on
#4404 to have no impact
on the TPS throughput of spacetimedb.
**This PR shouldn't be merged before #4404 has.**

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing

This tweaks a bench test.
Centril pushed a commit that referenced this pull request Mar 3, 2026
Re-open #4140 (reverted in #4292).


The original patch was merged a bit too eagerly.
It should go in _after_ 2.0 is released with some confidence.
@kim kim deleted the kim/commitlog/append-commit-reopen branch March 5, 2026 09:50
Ludv1gL added a commit to Ludv1gL/SpacetimeDB that referenced this pull request Apr 2, 2026
…r triggering

Two fixes:

1. Sync query methods (query_as_of_bsatn, query_as_of_pk_bsatn,
   query_time_series_bsatn, query_time_series_pk_bsatn) now detect
   missing/repairing segments and trigger repair via fire-and-forget
   spawns on the temporal runtime. Previously these methods had no
   health checks, so enabling temporal on existing databases would
   return "No data found" for AS OF queries via SQL without ever
   triggering repair. ALL BETWEEN worked because its async path
   had full health checks.

2. Grid alignment checks in build_ops.rs moved inside per-transaction
   loops for future-proofing when callers start batching multiple
   transactions per commit (upstream PR clockworklabs#4404 removed the n=1 limit).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants